It is known that neural networks have the problem of being over-confident when directly using the output label distribution to generate uncertainty measures. Existing methods mainly resolve this issue by retraining the entire model to impose the uncertainty quantification capability so that the learned model can achieve desired performance in accuracy and uncertainty prediction simultaneously. However, training the model from scratch is computationally expensive and may not be feasible in many situations. In this work, we consider a more practical post-hoc uncertainty learning setting, where a well-trained base model is given, and we focus on the uncertainty quantification task at the second stage of training. We propose a novel Bayesian meta-model to augment pre-trained models with better uncertainty quantification abilities, which is effective and computationally efficient. Our proposed method requires no additional training data and is flexible enough to quantify different uncertainties and easily adapt to different application settings, including out-of-domain data detection, misclassification detection, and trustworthy transfer learning. We demonstrate our proposed meta-model approach's flexibility and superior empirical performance on these applications over multiple representative image classification benchmarks.
translated by 谷歌翻译
梯度下降(GDA)方法是生成对抗网络(GAN)中最小值优化的主流算法。 GDA的收敛特性引起了最近文献的重大兴趣。具体而言,对于$ \ min _ {\ mathbf {x}} \ max _ {\ mathbf {y}} f(\ mathbf {x}; \ m m缩y} $以及$ \ mathbf {x} $,(lin等,2020)中的nonConvex证明了GDA的收敛性,带有sptepize的比率$ \ eta _ {\ mathbf {y}}}}/\ eta _ { }} = \ theta(\ kappa^2)$ with $ \ eta _ {\ mathbf {x}} $和$ \ eta _ {\ eta _ {\ mathbf {y}} $是$ \ mathbf {x}} $和$ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \\ Mathbf {y} $和$ \ kappa $是$ \ mathbf {y} $的条件号。尽管该步骤大比表明对最小玩家进行缓慢的训练,但实用的GAN算法通常对两个变量采用类似的步骤,表明理论和经验结果之间存在较大差距。在本文中,我们的目标是通过分析常规\ emph {nonconvex-nonconcave} minimax问题的\ emph {local contergence}来弥合这一差距。我们证明,$ \ theta(\ kappa)$的得分比是必要且足够的,足以使GDA局部收敛到Stackelberg equilibrium,其中$ \ kappa $是$ \ mathbf {y} $的本地条件号。我们证明了与匹配的下限几乎紧密的收敛速率。我们进一步将收敛保证扩展到随机GDA和额外梯度方法(例如)。最后,我们进行了几项数值实验来支持我们的理论发现。
translated by 谷歌翻译
如果对准确的预测的置信度不足,则选择性回归允许弃权。通常,通过允许拒绝选项,人们期望回归模型的性能会以减少覆盖范围的成本(即预测较少的样本)的成本提高。但是,正如我们所显示的,在某些情况下,少数子组的性能可以减少,同时我们减少覆盖范围,因此选择性回归可以放大不同敏感亚组之间的差异。在这些差异的推动下,我们提出了新的公平标准,用于选择性回归,要求每个子组的性能在覆盖范围内降低。我们证明,如果特征表示满足充分性标准或为均值和方差进行校准,则与所提出的公平标准相比。此外,我们介绍了两种方法,以减轻子组之间的性能差异:(a)通过在高斯假设下正规化有条件相互信息的上限,以及(b)通过对条件均值和条件方差预测的对比度损失正规。这些方法的有效性已在合成和现实世界数据集上证明。
translated by 谷歌翻译
我们研究基于梯度的随机近似问题的甲骨文复杂性。尽管在许多设置中,最佳算法和紧密的下界因这些问题而闻名,但在实践中使用时,这些最佳算法并不能达到最佳性能。我们通过关注实例依赖性复杂性而不是最坏情况的复杂性来解决这个理论实践差距。特别是,我们首先总结了已知的实例依赖性复杂性结果,并将它们分为三个级别。我们确定不同级别之间的支配关系,并提出了主导现有的第四个实例依赖性界限。然后,我们提供了足够的条件,根据该条件,具有时刻估计的自适应算法可以在不知道噪声水平的情况下达到拟议的结合。我们提出的算法及其分析为矩估计的成功提供了理论上的理由,因为它可以提高实例复杂性。
translated by 谷歌翻译
Machine Translation (MT) system generally aims at automatic representation of source language into target language retaining the originality of context using various Natural Language Processing (NLP) techniques. Among various NLP methods, Statistical Machine Translation(SMT). SMT uses probabilistic and statistical techniques to analyze information and conversion. This paper canvasses about the development of bilingual SMT models for translating English to fifteen low-resource Indian Languages (ILs) and vice versa. At the outset, all 15 languages are briefed with a short description related to our experimental need. Further, a detailed analysis of Samanantar and OPUS dataset for model building, along with standard benchmark dataset (Flores-200) for fine-tuning and testing, is done as a part of our experiment. Different preprocessing approaches are proposed in this paper to handle the noise of the dataset. To create the system, MOSES open-source SMT toolkit is explored. Distance reordering is utilized with the aim to understand the rules of grammar and context-dependent adjustments through a phrase reordering categorization framework. In our experiment, the quality of the translation is evaluated using standard metrics such as BLEU, METEOR, and RIBES
translated by 谷歌翻译
The devastation caused by the coronavirus pandemic makes it imperative to design automated techniques for a fast and accurate detection. We propose a novel non-invasive tool, using deep learning and imaging, for delineating COVID-19 infection in lungs. The Ensembling Attention-based Multi-scaled Convolution network (EAMC), employing Leave-One-Patient-Out (LOPO) training, exhibits high sensitivity and precision in outlining infected regions along with assessment of severity. The Attention module combines contextual with local information, at multiple scales, for accurate segmentation. Ensemble learning integrates heterogeneity of decision through different base classifiers. The superiority of EAMC, even with severe class imbalance, is established through comparison with existing state-of-the-art learning models over four publicly-available COVID-19 datasets. The results are suggestive of the relevance of deep learning in providing assistive intelligence to medical practitioners, when they are overburdened with patients as in pandemics. Its clinical significance lies in its unprecedented scope in providing low-cost decision-making for patients lacking specialized healthcare at remote locations.
translated by 谷歌翻译
Neural Architecture Search (NAS) is an automatic technique that can search for well-performed architectures for a specific task. Although NAS surpasses human-designed architecture in many fields, the high computational cost of architecture evaluation it requires hinders its development. A feasible solution is to directly evaluate some metrics in the initial stage of the architecture without any training. NAS without training (WOT) score is such a metric, which estimates the final trained accuracy of the architecture through the ability to distinguish different inputs in the activation layer. However, WOT score is not an atomic metric, meaning that it does not represent a fundamental indicator of the architecture. The contributions of this paper are in three folds. First, we decouple WOT into two atomic metrics which represent the distinguishing ability of the network and the number of activation units, and explore better combination rules named (Distinguishing Activation Score) DAS. We prove the correctness of decoupling theoretically and confirmed the effectiveness of the rules experimentally. Second, in order to improve the prediction accuracy of DAS to meet practical search requirements, we propose a fast training strategy. When DAS is used in combination with the fast training strategy, it yields more improvements. Third, we propose a dataset called Darts-training-bench (DTB), which fills the gap that no training states of architecture in existing datasets. Our proposed method has 1.04$\times$ - 1.56$\times$ improvements on NAS-Bench-101, Network Design Spaces, and the proposed DTB.
translated by 谷歌翻译
Deep learning-based object detection is a powerful approach for detecting faulty insulators in power lines. This involves training an object detection model from scratch, or fine tuning a model that is pre-trained on benchmark computer vision datasets. This approach works well with a large number of insulator images, but can result in unreliable models in the low data regime. The current literature mainly focuses on detecting the presence or absence of insulator caps, which is a relatively easy detection task, and does not consider detection of finer faults such as flashed and broken disks. In this article, we formulate three object detection tasks for insulator and asset inspection from aerial images, focusing on incipient faults in disks. We curate a large reference dataset of insulator images that can be used to learn robust features for detecting healthy and faulty insulators. We study the advantage of using this dataset in the low target data regime by pre-training on the reference dataset followed by fine-tuning on the target dataset. The results suggest that object detection models can be used to detect faults in insulators at a much incipient stage, and that transfer learning adds value depending on the type of object detection model. We identify key factors that dictate performance in the low data-regime and outline potential approaches to improve the state-of-the-art.
translated by 谷歌翻译
Online Social Networks have embarked on the importance of connection strength measures which has a broad array of applications such as, analyzing diffusion behaviors, community detection, link predictions, recommender systems. Though there are some existing connection strength measures, the density that a connection shares with it's neighbors and the directionality aspect has not received much attention. In this paper, we have proposed an asymmetric edge similarity measure namely, Neighborhood Density-based Edge Similarity (NDES) which provides a fundamental support to derive the strength of connection. The time complexity of NDES is $O(nk^2)$. An application of NDES for community detection in social network is shown. We have considered a similarity based community detection technique and substituted its similarity measure with NDES. The performance of NDES is evaluated on several small real-world datasets in terms of the effectiveness in detecting communities and compared with three widely used similarity measures. Empirical results show NDES enables detecting comparatively better communities both in terms of accuracy and quality.
translated by 谷歌翻译
Community detection in Social Networks is associated with finding and grouping the most similar nodes inherent in the network. These similar nodes are identified by computing tie strength. Stronger ties indicates higher proximity shared by connected node pairs. This work is motivated by Granovetter's argument that suggests that strong ties lies within densely connected nodes and the theory that community cores in real-world networks are densely connected. In this paper, we have introduced a novel method called \emph{Disjoint Community detection using Cascades (DCC)} which demonstrates the effectiveness of a new local density based tie strength measure on detecting communities. Here, tie strength is utilized to decide the paths followed for propagating information. The idea is to crawl through the tuple information of cascades towards the community core guided by increasing tie strength. Considering the cascade generation step, a novel preferential membership method has been developed to assign community labels to unassigned nodes. The efficacy of $DCC$ has been analyzed based on quality and accuracy on several real-world datasets and baseline community detection algorithms.
translated by 谷歌翻译